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[57] ABSTRACT 

A method and apparatus for forming a synthesized optical 
image with all of the desired objects in focus. A prescribed 
number of optical images of the same scene are provided by 
recording an optical image at a respective focal distance and 
changing the focal distance for each additional optical image 
as recorded. The optical images are subjected to wavelet 
transformation to form a multi-resolution representation. 
The coefficients of the various multi-resolution representa- 
tions are compared at the same position to detect the 
maximum spectral amplitudes. Based on the detected maxi- 
mum spectral amplitudes, another multi -resolution represen- 
tation is obtained. Then, an inverse wavelet transformation 
is performed to obtain the synthesized optical image. 
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METHOD AND DEVICE FOR 
SYNTHESIZING ALL-OBJECTS-IN-FOCUS 
IMAGES 

This invention concerns a method for synthesizing opti- 
cal images. More specifically, this invention concerns a 
method for synthesizing multiple optical images obtained by 
changing the focal distance. 

BACKGROUND OF THE INVENTION 

When a camera is used to record optical images and 
objects relatively far from the camera lens are in focus, 
objects nearer to the camera lens are out of focus and their 
recorded images are blurred. On the other hand, when 
focusing on tiie nearer objects, the farther objects are out of 
focus and their recorded images are blurred. This problem is 
believed to be related to the intrinsic characteristics of the 
optical system. According to a basic optical principle, a 
pinhole lens has an infinite focal depth, and all objects are 
in focus when their images are recorded within the pinhole 
lens of a pinhole camera. However, for the [ideal] pinhole 
lens, the hole is so small that not even a single photon can 
pass through it, and hrace no image is formed. That is, it is 
merely a hypothetical lens. 

Due to this basic physical limitation, the lens has a finite 
focal depth, and the picture of the objea is taken by means 
of automatic or manual focusing. However, when two 
objects, one nearer the camera and the other farther away 
from it, are to be photographed in the same picture, it is 
impossible to bring both into focus at the same time. That is, 
while bringing one object into focus, the other goes out of 
focus and its image becomes blurred. This is a disadvantage. 

Although not a technology for forming a synthesized 
image, a technology known as inverse filtering has been 
tested which attempts to create a clear image from the 
blurred portion of a picked-up image by means of enhancing 
the blurred portion. However, since it also amplifies noise, 
it fails to reach the stage of practical applications. 

OBJECT OF THE INVENTION 

An object of this invention is to provide a method for 
forming a synthesized image in which multiple optical 
images picked up at different focal distances are synthesized, 
and all of the objects are in focus in the synthesized image. 

SUMMARY OF THE INVENTION 
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This invention provides a method for obtaining a synthe- 
sized image while bringing objects at different positions into 
focus by means of digital signal processing using multiple 
image representations. The method for synthesizing optical 
images in accordance with the present invention comprises 
the following steps: a step in which firom the various image 
element signals of the multiple optical images added at 
different focal distances, wavelet transformation is per- 
formed to obtain multi-resolution representations witii a 
number corresponding to the number of the aforementioned 
optical images; a step in which the various coefficients of the 
aforementioned multi-resolution representations corre- 
sponding to tiie aforementioned optical images are com- 
pared at the same position, so as to detect the maximum ^ 
spectral an^licude at each of the prescribed positions; a step 
in which the synthesized multi-resohition representation is 
derived based on the maximum spectral amplitude detected 
at each of the aforementioned prescribed positions; and a 
step in which the image element signals are formed from the 
aforementioned synthesized multi-resolution representation 
by means of invrase wavelet transforraatioa 



65 



BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a flow chart illustrating an embodiment of a 
method for synthesizing optical images in accordance with 
the present inventioa 

FIG. 2 is a block diagram illustrating an embodimem of 
an apparatus for implementing the method for synthesizing 
optical images in accordance with the present invendon. 

FIG. 3 is a blodi diagram illustrating another embodiment 
of an apparatus for implementing the method for syntiiesiz- 
ing optical images in accordance with the present invention. 

FIG. 4 is a schematic diagram illustrating an embodiment 
for implementing multi-resolution representation in accor- 
dance with the invention. 

FIG. 5 is a photograph taken with a camera lens system 
picking up and focusing on a nearby object 

FIG. 6 is a photograph of the same scene shown in FIG. 
5, but with the camera lens system picking up and focusing 
on a remote object 

FIG. 7 is a diagram illustrating a multi-resolution repre- 
sentation of the optical image shown in FIG. 5 as an 
embodiment of this invention. 

FIG. 8 is a diagram illustrating a multi-resolution repre- 
sentation of the optical image shown in FIG. 6 as. an 
embodiment of this invention. 

FIG. 9 is a synthesized photograph of the same scene 
shown in FIGS. 5 and 6, but illustrating a synthesized optical 
image in accordance with the present invention, 

FIG. 10 is a diagram illustrating a portion of the program 
in an embodiment of the forward transformation of a multi- 
resolution representation in accordance with the invention. 

FIG. 11 is a diagram illustrating a portion of the program 
in an embodiment of the forward transformation of a multi- 
resolution representation in accordance with the invention. 

FIG. 12 is a diagram illustrating a portion of the program 
in an embodiment of the inverse transformation of a multi- 
resolution representation in accordance with the invention. 

FIG. 13 is a diagram illustrating a portion of the program 
in an embodiment of the inverse transformation of a multi- 
resolution representation in accordance with the invention. 

In reference numerals as shown in the drawings: 

10, image device 

20, pickup device 

30, synthesis device 

40, frame memory 

1, synthesized image 

DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

FIG. 1 is a flow chart illustrating an embodiment of a 
method for synthesizing an optical image in accordance with 
the invention. First of all, multiple optical images are formed 
at respective different focal distances by changing the focal 
distance of the camera lens (step 1). It is possible to adjust 
the focal distance either linearly or nonlinearly in the range 
from infinity to zero. However, there is virtually no problem 
in specifying the focal distance itself. It is only necessary to 
focus to the extent to which all objects to be brought into 
focus can be extracted. An appropriate number of images are 
pidced up in this step. 

Then, the picked-up images are transformed to the multi- 
resolution representation (step 2). This multi-resolution rep- 
resentation is based on the multi-resolution technology 
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discussed receatly in mathematics (see: C H. Chui: "An 
imrodncdon to wavelets;" I. DaubecMes: **Oithononnal 
bases of compactly supported wavelets." Communication of 
pure and ^lied mathematics. Vol. XU 909-996 (1988)). 
GeneraUy speaking, it is an isolation technology, in which, 5 
by means of special filtering and subsampling. the original 
signals arc isolated into high-frequency and low-frequency 
components without increasing the total amount of data. A 
prominent feature of this isolation technology is that the 
aforementioned low-frequency component can be further 
isolated using the same filtering method. In this way, the 
operation can be repeated for the required numbo- of cycles 
far the specific application. The layered signal representa- 
tion in the frequency region generated in.this way is usually 
called multi-resolution representation. 

Hien, the transformed multi-resolution representations 
are synthesized to form another new multi-resolution rep- 
resentation (step 3), followed by inverse multi-resolution 
representation to form the synthesized image (step 4). That 
is, the multiple multi-resolution representations correspond- ^ 
ing to the images in focus at different distances are synthe- 
sized to form another multi-resolution representation, which 
is then subjected to an inverse transformation to obtain a 
synthesized image in which the various objects are optimally 
in focus. ^ 

FIG. 2 is a diagram illustrating an example of the image 
equipment apparatus for implementing the method of this 
invention. In this example, the method and equipment for 
forming a synthesized image I for which optimal focusing is 
realized for all of the objects, from object A, nearest to the 39 
image equipment 10 to object Z, farthest away from it 
Pickup device 20 first focuses on object A (with the shortest 
focal distance) and its optical image (image I^) is recorded 
via a CCD (not shown in HG. 2) in frame memory 40. Then, 
the pickup device 20 focuses on object B (with the focal 35 
distance increased slightly), and its image is recorded in 
another frame memory (or on another region in the same 
frame memory). In the same way. the operation is performed 
in order until focusing of the most remote object Z (with 
focal distance up to infinity) is performed, and its image I^ 40 
is recorded in the corresponding firame memory. (For the 
various objects, there is no need to record A in I^, B in I^ 
etc., they may be matched to I„ (n= A, . . . , Z) in any [order]. 
However, in the present example, it is assumed that A is 
focused to form I4, B is focused to form I^ . . ,) Focusing 45 
for objects A-Z is carried out by means of the control signals 
from synthesis device 30. However, it may also be per- 
formed manually or by means of other control equipment 
The order in which the images are picked up can be selected 
at will. The method to be explained later is used to synthe- 50 
size images I^-I^ recorded in the various frame memories by 
means of synthesis device 30, forming synthesized image I. 
The memory may be in the form of a magnetic memory or 
an optical memory. It is also possible to input the picked-up 
signals into synthesis device 30. Also, photoelectric conver- 55 
sion equipment other than a CCD may be used as well. 

FIG. 3 illustrates another embodiment of the apparams. It 
differs from the case shown in FIG. 2 in that it has multiple 
pickup devices. For simplicity, only two pickup devices 20 
and 25 are shown in FIG. 3. The input optical mformation 60 
is equally divided between half-mirror 40 and mirror 50, and 
the portions are input into pickup device 20 and pickup 
device 25, respectively. Consequcntiy, one of the pickup 
devices 20 or 25 focuses on object A (with a shorter focal 
distance) to add image I^, which is then recorded in &ame 65 
memory 40. Then the other pickup device focuses on object 
B (with a longer focal distance) to capture image which 
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is recorded in frame memory 45. Then, by using a method 
to be explained lata-, an image I, which corresponds to the 
optimal focusing for both objects A and B, is formed by 
means of a synthesis device 30. 

In the fallowing, the multi-resolution representation using 
the wavel^ coefficients executed by synthesis device 30 will 
be explained. As shown in FIG. 4. image element signal A 
from the frame memory is isolated into four wavelet trans- 
formations a, b, c, and d by means of a hierarchical filter 
bank. Filters H and G correspond to high-pass and low-pass 
filters, respectively, and are complementary to each other 
(minor filters). Hlters G and H are called wavelet filters 
when filtering and subsampling (blocks indicated by annows) 
to form the orthogonal isolation of the input signal. FIG. 4 
is a diagram illustrating the case of 3-layer isolation of the 
signal by means of one-dimensional (in either the horizontal 
or vertical direction) filtering. For the 2-dimensiona] filter- 
ing adopted in this invention, sequential filtering is per- 
formed in the horizontal and vertical directions. Thai is, after 
execution of filtering in the horizontal direction (or vertical 
direction), filtaing in the vertical direction (or horizontal 
direction) is executed in this configuration. 

Although a frequency domain representation, the multi- 
resolution nqjrescntation also has spatial information (infor- 
mation related to position). On the other hand, for the 
Fourier transform, although a good representation in the 
frequency domain can be realized, no spatial information 
can' be preserved; hence, the position cannot be specified 
(the uncertainty principle). 

Next, the synthesis of multi-resolution images by means 
of multi-resolution representation will be explained. The 
multi-resolution representation using wavelet transforma- 
tion is a representation of the input signal in the fiequency 
domain. This representation also represents spatial informa- 
tion at the same time. In the multi-focus synthesis method, 
N multiple input images recorded for the same scene fol- 
lowed by wavelet transformation arc the objects, and the 
wavelet coef&dents that represent the most intense highr 
frequency component at each image element position are 
selected, since they are taken as the representation of the 
focal image at the aforementioned position. This process is 
executed for all of the image element positions of Uie image. 
By means of synthesis of the image elements selected for 
one multi-resolution representation detected at each posi- 
tion, a new multi-resolution representation can be formed 
with all of the image elements optimally focused. If needed, 
the aforementioned process may be not implemented for the 
lowest position at which the wavelet coefficients have 
almost the same value. 

On the other hand, for the aforementioned synthesis, it is 
necessary- to evaluate the spectral distribution of the image 
by changing die focal distance. Consequently, the Fourier 
transform of the overall image or partial small blocks (such 
as the cosine transform method used as a basic technology 
of image compression) may be considered. However, in this 
approach, the spectral lines and ends of lines usually expand 
into multiple coefficients, and this phenomenon makes the 
frequency and position information undeat Consequently, 
in the wavelet transformation, these two types of informa- 
tion are preserved, and hence it is basically appropriate for 
use in the multi-focus synthesis method. 

FIG. 5 is a photograph which illustrates the optical image 
provided by focusing on a nearby cup. FIG. 6 is a photo- 
graph which illustrates the optical image provided by focus- 
ing on a remote tree in the same scene shown in FIG. 5. 
FIGS. 7 and 8 show the multi-resolution representations for 
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the images of FIGS. 5 and 6, respectively. For FIGS. 5 and 
6, the original image is represented by means of 704 
pixelsx480 lines. They are isolated into 5 frequency bands 
shown in FIGS. 7 and 8. In the 2-dimensional muM- 
resolution representation, the lower right ooiresponds to the s 
higher frequency components. TTie lowest layer (the upper 
left comer) corresponds to the lowest frequency compo- 
nents, and it can be taken as the image which can pass 
through the lowest low-pass filter (and subsampled). 

In the case of multi-focus synthesis, the scenes of the 
same image are photographed for the desired number of 
cycles by a camera (or nmltiple camems) at different focal 
distances. However, for the lowest layer generated by the 
lowest low-pass filtering operation, even when photograph- 
ing at different focal distances, the images obtained are still i^ 
almost identical. (In other words, it is preferred that the 
number of layers be set to make the lowest frequency layers 
almost identical). Hie lowest frequency layer represents the 
mean brightness of the image. 

20 

On the other hand, for the higher frequency layers, the 
higher spectral components appear at various positions in 
the multi-resolution representation, and they correspond 
wcD to the objects in focus at the various focal distances. 

Consequently, when the multi-resolution representation is ^ 
adopted, the following takes place: 

(1) In the lowest frequency layer (the 15 pixelx22 lines on 
the upper left comer in FIGS. 7 and 8), each coefi&cient is 
derived as an average of multiple input images, or by 
extraction from the coefficients of a prescribed image. It is 30 
used as the lowest frequency layer of the synthesized image. 

(2) For the upper layers, at each position, there is a 
multi-resolution representation that gives the maximum 
spectral amplitude (maximum absolute value). Thai is, for 
the multi-resolution representation that gives the maximum 35 
spectral amplitude at position of i, j, as the simplest method, 

mamj^ - . ■ ■ I V^i) (1) 

is used as the representation (N= number of the multi- 
resolution representation, that is, number of the optical 
images added). It corresponds to the optimal focusing of the 
input scene, and the corresponding multi-resolution coefiS- 
cients give the informatioa The overall multi-resolution 
representation is formed according to this rule. In this way, 
at all of the positions, the multi-resolution representations 
that give the maximum spectral amplitude are detected, and 
they are synthesized to form the new multi-resolution rep- 
resentation. This mle itself is believed to have various forms. 
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Then, based on the detected maximimi spectral amplitude 
position, an inverse multi-resolution transformation is per- 
formed so that the synthesized multi-resolution representa- 
tion is transformed to an optical image composed of image 
element signals. As shown in FIG. 9, this optical image is a 
synthesized image of a neaxby cup and a far away tree, both 
of which arc well in focus in the picture. 

Although not directiy related to this invention, depending 
on the method of synthesis, there are cases in which the 
frame sizes may differ. This is an influence of the zoom 
effect as related to focusing of the various optical images. If 
needed, the size may be changed using various adjustment 
methods, such as "enlargement," "contraction," "interpola- 
tion," "extraction," etc., before the synthesis operation. 

FIGS. 10-13 illustraie portions of the program for the 
forward and inverse wavelet multi-resolution transforms. 

The aforementioned synthesis method does not depend on 
the image techniques related to content, such as edge extrac- 
tion, texture analysis, etc. The present method for synthe- 
sizing is completely different from methods with any pre- 
condition related to 3-dimensional objects. It is a method in 
which a stable synthesized result can be obtained with 
respect to all natural images. (That is, there is absolutely no 
generation of significant degradation in the case when the 
precondition is violated.) In addition, the edge analysis is 
still in the stage of initial devclopmenL When the number of 
edges is increased, the processing time increases exponen- 
tially. Conscquentiy, it has not been used in practical appli- 
cations. At present, the present method is the most appro- 
priate method for practical applications for namral images. 

Also, since it is based on detection of the maximum 
spectral amplitude from the multiple multi-resolution coef- 
ficients, there is no limitation to the number of optical 
images which can be added. By capturing multiple optical 
images, it is possible to obtain a synthesized image that is in 
focus at almost all positions. Also, enlargement is possible 
to obtain very correct 3-D information (distance, appearance 
of object, etc.) 

In the above description, embodiments of this invention 
have been presented. However, this invention is not limited 
to these embodiments. This invention can be used with 
various types of equipment, such as cameras, television 
cameras, monitoring cameras, medical imaging cameras, 
etc. 

This invention provides an effective method for forming 
a single synthesized optical image from multiple optical 
images added at different focal distances, with optimal 
focusing. 
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t* Forward Wavclei Ttansfann */ 

/* This subroutine performs the N=3 Daubichics */ 

/* forward wavelet transfonn on image deflned ♦/ 

/* es global variable "short YYcomponcni". */ 

/♦ The transformed resulu ait stored back in */ 

f* "YYcomponent". V 

f* */ 

r 11/20/1992 V 

f* Texas Insiraments video/SIS/TRDC (Yamaguchi) */ 



ForwardWavelei (short YYconq»onem [41 (500] [720], int Vsize, int Hsi3£, im layers, im component ) 
{ 

sboit TEMPcomponeat [300] [720]; 
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APPENDIX A-conlinued 



double HCoefR 1 1 HOOm 0332670532950. 0.8068915093U. a459877502118. 

-0.13501 1020010. -a085441273882. 0.035226291882, 0}; 
/• ITns is DanMchics filter of N=3 */ 
doubk GCocmilHO. 0.035226291882. a0854412738S2, -0.115011020010.-0.459877502118, 
0.806891509311, -a33267Q55295a 0, 0, 0, OJ; 

register ioAc 
inlintten^l 
intrcfV.icfH; 
int Fesolution; 

£or(i=0-.t<ll;i++) 
{ 

HCocmil^=sqrt (2.0); 
GCoemil/=sqrt (ZO); 

} 

for (re8olutioii=0;tesoltttion<=layer3-l?«oliiiioihH-) 
{ 

for (TtfV=0; rBfV<Vsire?efV++) for (fe£H=0; rcfH<Hsizc;iefH4«=2) 
{ 

for (i=-5-4<^3++) 
{ 

if (iefH+i<0) intieiniH=YYcoinponcm locmponentl [rcfVl HrefH+i)l*HCceffl5-i|; 
else if ((refH+i)>=(H5izc-I)) 

mltcmi>t=YYcoinponcm (component) IiefV) ICHsize-I)*2-(rcfH+i)]»HCocffl5Hil; 
else iiittemj>4=YYconq»ncnt [component) |refV] [refH+i)*HCoeffI5^); 

} 

TEMPconqwncnt [refVUrefH/2)=inttenq>; 

intten^j^^^ 
for (i^5p<=5p-H-) 
{ 

if (lefH+icO) mtteii5>t=:Yycomponent (oonqwneat) [refV) HrefH+i)]*GCoeffI5-i]; 

iiittcrap+=YYcompoiient [componem) IrefV) |CHsizB-l)*2-(refH+i)]*GCoeffI5-i]; 
else ijittenip+=YYcoiiiponcnt (component] (refV) lTBfH+i)*GCoeffl5-i); 

} 

TEMPcoTtqwnent (rcfVl(Hsiic/2+refH/2J=intieinp; 

} 

for (refV=0-^fV<Vsize3cfV-Ka2) for (refH=<J?efH<Hsize?^++) 
{ ^ 
initeinp=0; 
for (i=-5p<=S3++) 
{ 

if (refV+i<0) iimein{H<=TEMPconfiponcnt (-(refV+i)] [icfHrHCocfiFI5-i); 
else if ((refV+i)>=(Vsi2e-l)) 

imtBm;>f=TEMPcomponent |(V»ize-l)*2-(rBfV+i)) [fcfHl*HCooffI5-Hl; 
else iimcmpt=TEMPcomponent [icfV+il (rcfH]*HCoeff[5-l): 

> 

YYconq)onent [consponcnt] (rcfV/2) (refH)=iimcnq); 

inttemp=0; 
for 0=^53C=53++) 
{ 

if (rcfV+i<0) inttcnqH=TEMPconq»nent|-{refV+0] (relH]*OCocffl5-il; 
else if ((refV+i>=(Vsi2e-l)) 

inttciiq>+«TEMPcoii^ncnH(Vsi2e-l)^refV-ri)] lrefHl*GCocffl5-i); 
else inttemiM=a'EMPcoiiq)onem [refV4i] (iefH)*GCDcf55-i); 

} 

YYcompoiiciu {component) [V5izB/2+rerV/2) lrEEH]=intieny; 

} 

Hsiz^Hsizc/2; Vsir;^Vsizc/2; 

} 

} 
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APPENDIX B 



f* Inverse Wavdet Tnuufoxm 


•/ 


/* This loolins pcrfomis the N=3 Danbidues 


•/ 


f* inverse wavelet tnmsfbnn oh image deSned 


♦/ 


f* as gbbal variable "short YYconqKmenl" 


•/ 


t* The transfonncd results oic stcred back in 


♦/ 


/• "YYconqjoacm". 
1* 


•/ 
♦/ 


{* 7/30/1992 


•/ 


t* Texas iTiMrnTncntt video/SIS/TRDC (Ysmagnchi) 


•/ 




InverseWavdet (short YYcorrqKment |4] (500) [720], int Vazc, int Hsize, im layers, int component) 



{ 

double HCoefiinHO,0,aO,0, 0.332670552950, 0.806891509311, 0.459877502118, 
-0.135011020010. -O.08544l273882.aO3522629l882); 
/* This is Daabichies filter of N=0 */ 
doable Gcoeffl 1 1 HO. 0.035226291 882, 0,085441273882, -0.13501 1020010, -0.4598775021 1 8, 
O.S0689150931 1 . -0.33267055295a 0,0,0,0}; 

register i j,k; 

int imtcmpl inncinp2; 

int refV, refH; 

int resolutioo; 

short YYlempl 1500] [720]; 
short YYtemp2 (5001 [720); 

for (i=0-4<ll;i++) 
{ 

HCoeffli]*=sqn(ZO); 
GCoe8Ti]*=sqn(2.0); 

} 

for CrBsduiioiF:<)^esolutiai)<?=laycr5-l3BsolutioiH-+) 
{ 

for (rcfV=03tfV<Vsize*23cfV++) for (rcfH=0;rcfH<Hsize3rfH-H-) 
{ 

YYtempl ticfV) |icfH*2}=YYcomponcnt [component] [icfV] [rcfHl; 
YYiempl {rcfVl lrefH*2+l |=0; 

YYteinp2 [refvj jre£H*21=YYcompanent (compoaent) [refV] [Hsize+iefH]; 
YYtcinp2 [rcfV] [icfH*2+l|=0; 

} 

for (refV=0; rcfV<V5izc*2?icfV++) for (rc£K=0?efH<H$i2e*2uBffi-H-) 
{ 

intiempl=0; inttemp2=0; 
for (i=-5^<=a;i++) 
{ 

if(refH+i<0) 
{ 

inaempl+=YYiempl |rcfV] |-(rcfH+i)rHCocff[5f IJ; 
inttcmp2+=YYteinp2 (refVl |-<refH+i)l*GCocffl5+l]; 

} 

else if ((refH+i)>=(Hsize*2-l)) 
{ 

innempli=YYicmpl [rcfVl (CH$ize*2-l)*2-<reIH+i)rHCoeSI5+iI; 
intlcmp2+=YYtemii2 (refVI [(Hsize*2-l)*2-<refH+i)I*GCoeflfI5+il; 

} 

else 
{ 

iuttcrapl-H=YYtenq)l [refV] (rcfH+i]*HCoeffl5+i); 
inttcmp2-f=YYtcrap2 [rcfVl trefH+il'GCoeSlS+i]; 

} 

} 

YYcomponcnt [conqwnentl (refV] [refH]=inttempI+iiittemp2; 

} 

for (rEfV=07efV<Vii2c?cfV++) for (rcfH=0-,refH<Hsizc*27efH++) 
{ 

YYtcmpl IrcfV»2] |refH|=YYcomponcm [coraponem] (refV) Ircffll; 
YYtcmpl lrefV*2-»-l] irefHl=0; 

YYtcmp2 lrcfV»21 !icfHl=YYconq)oncm (compcncnll [Vaze+refVI (refHj; 
YYtemp2 (refV»2+ll [refH =0; 

} 

for (rEfV=:0?tfV<V$i2e*2^cfV++) for (refH=0-,refH<Ha2e*27efH++) 
{ 

iniiec9l=0; intten^)2=0; 
for 0=-5;<=5-4++) 
{ 

if (refV+i<0) 
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\ 

imtenqjl-f^YYtempl HrcfV-H*)I (refHi*HCoe£E|5+il; 
iiitteaq>2+=YYtcmp2 [-(itfV+oj IrcfHl*GCocfil5fi]; 

) 

else if ((refV40>=<Vsitt*2-l)) 

iattm^I+^YYtenipl ((Vsize^-irMrefV-H*)! (refH)*HCocSl5+il; 
iiiacaq»2^«YYicmp2 C(Vsi2e*2-ir2-(rcfV-ri)l |rcafl*GCocffi5+4I; 

} 

eke 

iiittBnq)l4«YYtciiq)l (refV+i) |itfHI*HCoeffi5Hl; 
initcaqj2-K=YYtcmp2 [irfV+ij ircfHl»GCoaffI5+ij; 

} 

> 

YYoon^nent {compooeiu) [lefV] [r^)=inttenq)l-Hnttenip2; 

} 

Hsixe*=2: VsiM»=2; 

> 



I claim: 

1. A method for synthesizing an all-objects-in-focus two- 
dimensional image of several objects at various correspond- 
ing object distances, comprising the steps of: 2S 

for each object capturing a carresponding pixel image 
focused at its object distance; 

transforming each pixel image into a corresponding multi- 
resolution representation expressed as a set of coeffi- 
cients; 30 

comparing, at each image position, the various coeffi- 
cients of the multi-resolution representations to detect 
the maximum spectral amplitude at each image posi- 
tion; 

35 

deriving a synthesized multi-resolution representation by 

using the coefficient having the maximum spectral 

amplitude at each image position; 
reverse-transforming the synthesized multi-resolution 

representation by inverse wavelet transfonnation into a 4q 

synthesized optimally focused image. 

2. A method for synthesizing an all-objects-in-focus two- 
dimensional image of a scene of N objects (O,, O2, O3. . . 
0;v) positioned at various corresponding object distances 
(dp d2, dj. . . djv), comprising: 45 

for each object capturing a corresponding two dimen- 
sional original pixel image 0^, I2, Is - . . I//) of the scene 
focused at its object distance; 

transforming each original pixel image by wavelet trans- 
formation to obtain a corresponding multi-resolution 50 
representation comprising a set of stored coefficents 

{A,/'\ Ag^^ Aif^) corresponding to image 

positions (ij) and spacial frequencies; 

for each image position (ij), selecting from among the N 
multi-resolution representations the coefficient of 
maximum spectral amplitude; 



combining the selected multi-resolutian coefficients into a 
synthesized multi-resolution representation A^^^"*^; 
and 

reverse-transforming the synthesized multi-resolution 
representation by a reverse multiple-resolution trans- 
formation to CTcate a synthesized optimally focused 
image. 

3. A device for synthesizing an all-objects-in-focus two 
dimensional image of a scene of N objects (Oj, Oj, O3. . . 
Of^) positioned at various corresponding object distances 
(dj, dj, dg. . . d;^^), comprising: 

a camera for capturing for each object a corresponding 
two dimensional original pixel image (Ij, I2, 13. . . 1^) 
of the scene focused at its object distance; 

a first filter for transforming each original pixel image by 
wavelet transformation into a corresponding multi- 
resolution representation comprising a set of stored 
coefficents (A/". A^^^^ .... A/'^ conespond- 
ing to image positions (i j) and spacial frequencies; 

a comparator for selecting for each image position (i j) 
from among the N multi-resolution representations the 
coefficient of maximum spectral amplitude; 

a synthesizer circuit for combining the selected multi- 
resolution coefficients into a synthesized multi-resolu- 
tion representation A^^^**^; and 

a second filter for reverse-transforming the synthesized 
multi-resolution representation by a reverse multiple- 
resolution transfonnation to create a synthesized opti- 
mally focused image. 

4* 4i ♦ « * 
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